{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/togethercomputer/together-cookbook/blob/main/Agents/KlavisAI/Use_Klavis_with_Together.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "# Together AI + Klavis AI Integration\n",
    "\n",
    "# <img src=\"../../static/togetherai-klavis.png\" width=\"700\">\n",
    "\n",
    "In this tutorial, we'll explore how to build an AI agent that integrates Together AI's powerful LLMs with Klavis MCP Servers, enabling seamless interaction with external services and APIs.\n",
    "\n",
    "This integration combines:\n",
    "- **Together AI**: High-performance open-source LLMs with function calling capabilities\n",
    "- **Klavis AI**: MCP (Model Context Protocol) servers for connecting to external tools and services\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "## Prerequisites\n",
    "\n",
    "Before we begin, you'll need:\n",
    "\n",
    "- **Together AI API key** - Get yours at [together.ai](https://together.ai/)\n",
    "- **Klavis AI API key** - Get yours at [klavis.ai](https://klavis.ai/)\n",
    "\n",
    "Make sure to keep these API keys secure and never commit them to version control!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m25.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m25.1.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# Install the required packages\n",
    "%pip install -qU together klavis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import json\n",
    "from together import Together\n",
    "from klavis import Klavis\n",
    "from klavis.types import McpServerName, ToolFormat\n",
    "\n",
    "# Set environment variables\n",
    "os.environ[\"TOGETHER_API_KEY\"] = \"your-together-api-key-here\"  # Replace with your actual Together API key\n",
    "os.environ[\"KLAVIS_API_KEY\"] = \"your-klavis-api-key-here\"      # Replace with your actual Klavis API key"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "## Create AI Agent with MCP Integration\n",
    "\n",
    "Now we'll create an intelligent agent that uses Together AI's powerful LLMs with Klavis MCP servers. This agent will:\n",
    "\n",
    "1. **Discover Tools**: Automatically find available tools from MCP servers\n",
    "2. **Function Calling**: Use Together AI's function calling capabilities\n",
    "3. **Tool Execution**: Execute tools through Klavis API\n",
    "4. **Smart Responses**: Generate intelligent responses based on tool results\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Agent:\n",
    "    def __init__(self, together_client, klavis_client, mcp_server_url, model=\"meta-llama/Llama-3.3-70B-Instruct-Turbo\"):\n",
    "        self.together = together_client\n",
    "        self.klavis = klavis_client\n",
    "        self.mcp_server_url = mcp_server_url\n",
    "        self.model = model\n",
    "        print(f\"\ud83e\udd16 Agent initialized with Together AI model: {self.model}\")\n",
    "    \n",
    "    def process_request(self, user_message):\n",
    "        # 1. Get available tools\n",
    "        mcp_tools = self.klavis.mcp_server.list_tools(\n",
    "            server_url=self.mcp_server_url,\n",
    "            format=ToolFormat.OPENAI,\n",
    "        )\n",
    "        \n",
    "        # 2. Call LLM with tools\n",
    "        messages = [\n",
    "            {\"role\": \"system\", \"content\": \"You are a helpful AI assistant with access to various tools.\"},\n",
    "            {\"role\": \"user\", \"content\": user_message}\n",
    "        ]\n",
    "        \n",
    "        response = self.together.chat.completions.create(\n",
    "            model=self.model,\n",
    "            messages=messages,\n",
    "            tools=mcp_tools.tools\n",
    "        )\n",
    "        \n",
    "        assistant_message = response.choices[0].message\n",
    "        messages.append(assistant_message)\n",
    "        \n",
    "        # 3. If LLM wants to use tools\n",
    "        if assistant_message.tool_calls:\n",
    "            \n",
    "            # Execute each tool call\n",
    "            for tool_call in assistant_message.tool_calls:\n",
    "                tool_name = tool_call.function.name\n",
    "                tool_args = json.loads(tool_call.function.arguments)\n",
    "                \n",
    "                print(f\"\ud83d\udee0\ufe0f Calling tool: {tool_name} with args: {tool_args}\")\n",
    "                # Call tool via Klavis SDK\n",
    "                tool_result = self.klavis.mcp_server.call_tools(\n",
    "                    server_url=self.mcp_server_url,\n",
    "                    tool_name=tool_name,\n",
    "                    tool_args=tool_args,\n",
    "                )\n",
    "                \n",
    "                messages.append({\n",
    "                    \"role\": \"tool\",\n",
    "                    \"tool_call_id\": tool_call.id,\n",
    "                    \"content\": str(tool_result)\n",
    "                })\n",
    "            \n",
    "            # 4. Get final response from LLM\n",
    "            final_response = self.together.chat.completions.create(\n",
    "                model=self.model,\n",
    "                messages=messages\n",
    "            )\n",
    "            return final_response.choices[0].message.content\n",
    "        \n",
    "        # If no tools needed, return the assistant message directly\n",
    "        return assistant_message.content\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "## Use Case 1: Summarize YouTube Video"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u2705 Created YouTube MCP instance\n",
      "\ud83e\udd16 Agent initialized with Together AI model: meta-llama/Llama-3.3-70B-Instruct-Turbo\n",
      "\ud83d\udee0\ufe0f Calling tool: get_youtube_video_transcript with args: {'url': 'https://www.youtube.com/watch?v=TG6QOa2JJJQ'}\n",
      "The YouTube video \"Together AI CEO: Open Source Is the Future of AI\" features an interview with Vipul Ved Prakash, the CEO of Together AI, and Bucky Moore, a partner at Kleiner Perkins. The discussion revolves around Together AI's $102.5 million Series A fundraise, led by Kleiner Perkins, and the company's focus on open-source AI.\n",
      "\n",
      "Here is a comprehensive summary of the video with timestamps:\n",
      "\n",
      "* 0:00 - Introduction to the video and the guests\n",
      "* 0:30 - Discussion of Together AI's Series A fundraise and the company's mission\n",
      "* 1:45 - Vipul Ved Prakash explains the importance of open-source AI and how it can benefit the industry\n",
      "* 3:10 - Bucky Moore shares his perspective on the potential of open-source AI and its applications\n",
      "* 4:30 - The guests discuss the challenges and opportunities in the AI industry, including the need for more diverse and inclusive data sets\n",
      "* 5:50 - Vipul Ved Prakash talks about the company's plans for the future and how it aims to make AI more accessible and affordable for everyone\n",
      "* 6:40 - Conclusion and final thoughts from the guests\n",
      "\n",
      "Overall, the video provides insights into the future of AI and the potential of open-source AI, as well as the company's plans and goals.\n"
     ]
    }
   ],
   "source": [
    "\n",
    "# Example YouTube video URL - replace with any video you'd like to analyze\n",
    "YOUTUBE_VIDEO_URL = \"https://www.youtube.com/watch?v=TG6QOa2JJJQ\"\n",
    "\n",
    "# 1. Initialize Together AI client and Klavis client\n",
    "together_client = Together(api_key=os.getenv(\"TOGETHER_API_KEY\"))\n",
    "klavis_client = Klavis(api_key=os.getenv(\"KLAVIS_API_KEY\"))\n",
    "\n",
    "# 2. Create YouTube MCP server instance\n",
    "youtube_mcp_instance = klavis_client.mcp_server.create_server_instance(\n",
    "    server_name=McpServerName.YOUTUBE,\n",
    "    user_id=\"1234\",\n",
    ")\n",
    "\n",
    "# 3. Create an agent with YouTube MCP server\n",
    "agent = Agent(\n",
    "    together_client=together_client, \n",
    "    klavis_client=klavis_client, \n",
    "    mcp_server_url=youtube_mcp_instance.server_url,\n",
    "    model=\"meta-llama/Llama-3.3-70B-Instruct-Turbo\"\n",
    ")\n",
    "\n",
    "# 4. Process the request\n",
    "response = agent.process_request(\n",
    "    f\"Please analyze this YouTube video and provide a comprehensive summary with timestamps: {YOUTUBE_VIDEO_URL}\"\n",
    ")\n",
    "\n",
    "print(response)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "## Use Case 2: Send Email\n",
    "\n",
    "**Note**: Gmail integration requires OAuth authentication, so you'll need to authorize the application in your browser.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u2705 Created Gmail MCP instance\n",
      "\ud83d\udd10 Opening OAuth authorization for Gmail\n",
      "If you are not redirected automatically, please open this URL: https://api.klavis.ai/oauth/gmail/authorize?instance_id=d9d482b3-433a-4330-9a8b-9548c0b0a326\n"
     ]
    }
   ],
   "source": [
    "import webbrowser\n",
    "\n",
    "# Create Gmail MCP server instance\n",
    "gmail_mcp_instance = klavis_client.mcp_server.create_server_instance(\n",
    "    server_name=McpServerName.GMAIL,\n",
    "    user_id=\"1234\",\n",
    ")\n",
    "\n",
    "# Redirect to Gmail OAuth page for authorization\n",
    "webbrowser.open(gmail_mcp_instance.oauth_url)\n",
    "print(f\"\ud83d\udd10 Opening OAuth authorization for Gmail\")\n",
    "print(f\"If you are not redirected automatically, please open this URL: {gmail_mcp_instance.oauth_url}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\ud83e\udd16 Agent initialized with Together AI model: Qwen/Qwen2.5-72B-Instruct-Turbo\n",
      "\ud83d\udee0\ufe0f Calling tool: send_email with args: {'body': 'This is a test email sent using the Together AI and Klavis AI integration. The email was sent automatically by your AI agent!', 'subject': 'Greetings from Together AI + Klavis Integration', 'to': ['zihaolin@klavis.ai']}\n",
      "The email has been sent successfully to zihaolin@klavis.ai with the subject 'Greetings from Together AI + Klavis Integration'. The email ID is 19776f818ce706db.\n"
     ]
    }
   ],
   "source": [
    "# Email configuration\n",
    "EMAIL_RECIPIENT = \"zihaolin@klavis.ai\"  # Replace with the recipient's email\n",
    "EMAIL_SUBJECT = \"Greetings from Together AI + Klavis Integration\"\n",
    "EMAIL_BODY = \"This is a test email sent using the Together AI and Klavis AI integration. The email was sent automatically by your AI agent!\"\n",
    "\n",
    "# After OAuth authorization is complete, create the Gmail agent\n",
    "gmail_agent = Agent(\n",
    "    together_client=together_client,\n",
    "    klavis_client=klavis_client,\n",
    "    mcp_server_url=gmail_mcp_instance.server_url,\n",
    "    model=\"Qwen/Qwen2.5-72B-Instruct-Turbo\"\n",
    ")\n",
    "\n",
    "# Send the email\n",
    "response = gmail_agent.process_request(\n",
    "    f\"Please send an email to {EMAIL_RECIPIENT} with the subject '{EMAIL_SUBJECT}' and the following body: '{EMAIL_BODY}'\"\n",
    ")\n",
    "\n",
    "print(response)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "## Summary\n",
    "\n",
    "This tutorial demonstrated how to integrate Together AI's function calling capabilities with Klavis MCP servers to create powerful AI applications. We covered two practical examples:\n",
    "\n",
    "**\ud83c\udfa5 YouTube Integration**: Built an AI assistant that can automatically summarize YouTube videos by extracting transcripts and providing detailed, timestamped summaries.\n",
    "\n",
    "**\ud83d\udce7 Gmail Integration**: Created an AI-powered email assistant that can send emails through Gmail with OAuth authentication.\n",
    "\n",
    "### Key Takeaways:\n",
    "- **Easy Setup**: Klavis MCP servers can be created with just a few lines of code using the official SDK\n",
    "- **Together AI Compatible**: All tools are formatted for seamless Together AI function calling\n",
    "- **Versatile**: Support for both simple APIs (YouTube) and OAuth-authenticated services (Gmail)\n",
    "- **Scalable**: The same pattern can be applied to any of the 100+ MCP servers available in Klavis\n",
    "\n",
    "### \ud83d\ude80 Next Steps\n",
    "- **Explore More MCP Servers**: Try other available servers like Slack, Notion, CRM etc.\n",
    "- **Experiment with Different Models**: Test various Together AI models for different use cases.\n",
    "- **Build Complex Multi-Server Workflows**: Create sophisticated agents that combine multiple services\n",
    "- **Production Deployment**: Scale these patterns for production applications\n",
    "- **Custom MCP Servers**: Build your own MCP servers for proprietary systems\n",
    "\n",
    "### \ud83d\udd17 Useful Resources\n",
    "- [Together AI Documentation](https://docs.together.ai/)\n",
    "- [Klavis AI Documentation](https://www.klavis.ai/docs/)\n",
    "- [MCP Protocol Specification](https://modelcontextprotocol.io/)\n",
    "- [Together AI Models](https://docs.together.ai/docs/inference-models)\n",
    "- [Klavis MCP Servers](https://www.klavis.ai/docs/mcp-servers)\n",
    "\n",
    "**Happy building with Together AI and Klavis!** \ud83d\ude80\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}